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SIMMARY 


Decision  making  requires  two  sequential  actions:  Assessing  what  one  wants 
and,  then,  figuring  out  what  one  can  get.  Considerable  research  has  gone 
Into  Improving  our  ability  to  carry  out  the  second  of  these  acts  through 
the  design  of  sophisticated  decision  models  and  calculation  procedures 
and  through  the  study  of  people's  ability  to  wake  the  required  jud^ents 
of  fact.  It  has  been  found,  for  example,  that  people  are  prone  to  syste¬ 
matic  biases  In  the  judpaents  of  probabilistic  events,  biases  which  limit 
the  validity  of  analyses  and  call  for  remedial  procedures. 

Typically,  however.  It  has  been  presumed  that  people  i-now  what  they  want 
and  that  with  a  little  care,  those  values  can  be  elicited.  Furthermore, 
those  values  are  sufficiently  well  developed  to  be  elicited  in  whatever 
fona  Is  most  convenient  for  the  ellcltor  (opinion  surveyor,  decision 
analyst,  etc.).  Research  reviewed  here  suggests  that  this  may  not  be 
the  case  and  that  a  fundamental  change  may  be  called  for  In  the  way  we 
study  and  elicit  values. 

When  faced  with  complex,  unfamiliar  Issues,  people  may  have  poorly 
formulated,  even  Incoherent,  values.  In  such  situations,  where  people 
do  not  know  what  they  want,  the  values  they  express  may  be  highly  labile. 
Subtle  changes  In  how  Issues  are  posed,  questions  are  phrased  and  responses 
are  elicited  can  have  marked  effects  on  people's  expressed  preferences. 

That  Is.  the  method  may  be  part  of  the  message  (the  reported  value). 

Formal  decision  making  methods  have  no  way  of  accomodating  these  effects, 
that  Is,  deciding  which  of  a  pair  of  contradictory  expressed  values  is  to 
be  believed.  However,  confronting  these  effects  Is  unavoidable  If  we  are 
to  elicit  values  at  all.  If  one  Is  Interested  In  what  an  Individual 
really  feels  about  a  values  Issue,  there  may  be  no  substitute  for  an 


Interactive  elicitation  procedure  which  acknowledges  the  ellcltor's  role 
In  helping  the  respondent  to  create  and  enunciate  values.  There  Is,  how¬ 
ever,  no  guarantee  that  even  the  greatest  of  care  will  resolve  fundamental 
Inconsistencies  In  values.  It  may,  at  times,  be  necessary  to  acknowledge 
that  we  do  not  know  what  we  want  and  structure  our  actions  accordingly. 


11 


ACKNOWLEDGEMENT 


This  research  was  supported  by  the  Advanced  Research  Projects  Agency  of 
the  Department  of  Defense  and  was  monitored  by  Office  of  Naval  Research 
under  Contract  N00014-79-C-0029  (ARPA  Order  No.  3668)  to  Perceptronlcs , 
Inc. 


1.  INTRODUCTION 


An  article  of  faith  among  students  of  value,  choice  and  attitude  judg¬ 
ments  Is  that  people  have  reasonably  well-defined  opinions  regarding  the 
desirability  of  various  events.  Although  these  opinions  may  not  be 
Intuitively  formulated  In  nunerlcal  (or  even  verbal)  form,  careful  ques¬ 
tioning  can  elicit  judgnents  representing  people's  underlying  values. 
From  this  stance  elicitation  procedures  are  neutral  tools,  bias-free 
channels  which  translate  subjective  feelings  Into  scientifically  usable 
expressions.  They  Impose  no  views  on  respondents,  beyond  focusing  atten 
tlon  on  those  value  Issues  of  Interest  to  the  Investigator. 

What  happens,  however.  In  cases  where  people  do  not  know,  or  have  diffi¬ 
culty  appraising,  what  they  want?  Under  such  circumstances  elicitation 
procedures  may  become  major  forces  In  shaping  the  values  expressed,  or 
apparently  expressed.  In  the  Judgnents  they  require.  They  can  Induce 
random  error  (by  confusing  the  respondent),  systematic  error  (by  hinting 
at  what  the  "correct"  response  is),  or  unduly  extreme  judgments  (by 
suggesting  clarity  and  coherence  of  opinion  that  are  not  warranted).  In 
such  cases,  the  method  becomes  the  message.  If  elicited  values  are  used 
as  guides  for  future  behavior,  they  may  lead  to  decisions  not  In  the 
decision  maker's  best  Interest,  to  action  when  caution  Is  desirable  (or 
the  opposite)  or  to  the  obfuscation  of  poorly  formulated  views  needing 
careful  development  and  clarification. 

The  topic  of  this  paper  Is  the  confrontation  between  those  who  hold 
(possibly  Inchoate)  values  and  those  who  elicit  values.  By  "values," 
we  mean  evaluative  Judgnents  regarding  the  relative  or  absolute  worth  or 
desirability  of  possible  events.  Such  events  may  be  general  (being 
honest)  or  specific  (winning  a  particular  lottery).  Their  consequences 
(or  outcomes)  may  have  one  or  many  salient  attributes  and  may  be 


certainties  or  possibilities.  Such  a  broad  definition  captures  just 
about  any  task  ever  Included  under  the  topics  of  value,  choice  or  prefer¬ 
ence,  as  well  as  many  that  would  fit  comfortably  under  attitudes,  opinions, 
and  decision  making.  Our  discussion  is  limited  to  situations  in  which 
people  are  reporting  their  values  as  honestly  as  possible;  the  further 
complication  of  measuring  values  in  the  face  of  strategic  behavior  Is 
not  considered. 

The  recurrent  theme  of  this  paper  Is  that  subtle  aspects  of  how  problems 
are  posed,  questions  are  phrased  and  responses  are  elicited  can  have  sub¬ 
stantial  impact  on  judgments  that  supposedly  express  people's  true  values. 
Furthermore,  such  lability  In  expressed  preferences  is  unavoidable:  ques¬ 
tions  must  be  posed  in  some  manner  and  that  manner  may  have  a  large  effect 
on  the  responses  elicited.  Pursuit  of  the  Issues  raised  here  can  at  best 
alert  ellcitor  and  respondent  to  such  impacts,  making  these  effects  deli¬ 
berate  rather  than  covert. 

One  might  hope  that  such  analysis  would  identify  the  “right"  way  to  ask 
about  values.  To  foreshadow  our  conclusions,  we  believe  that  the  quest 
for  a  right  way  is,  at  times,  ill-founded.  While  there  are  some  obvious 
pitfalls  to  avoid,  instability  Is  often  Inherent  In  our  values.  Rather 
than  trying  to  circumvent  such  lability,  we  should  try  to  exploit  the 
insight  It  provides  into  the  nature  of  values,  and  their  formation,  change 
and  application. 

1.1  When  and  How  People  Wight  Not  Know  What  They  Want 

People  are  most  likely  to  have  clear  preferences  regarding  Issues  that 
are  familiar,  simple  and  directly  experienced.  Each  of  these  properties 
is  associated  with  opportunities  for  trial-and-error  learning,  particularly 
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such  learning  as  may  be  sunmarlzed  In  readily  applicable  rules  or 
homilies.1  Those  rules  provide  stereotypic,  readily  justifiable 
responses  to  future  questions  of  values.  When  adopted  by  Individuals, 
they  may  be  seen  as  habits;  when  adopted  by  groups,  they  constitute  tra 
dltlons. 


The  acceptability  and  perceived  validity  of  such  adages  as  "honesty  Is 
the  best  policy"  and  "cleanliness  Is  next  to  godliness"  Is  to  some  extent 
appropriate.  As  guides  to  living,  they  have  been  subjected  to  some 
empirical  testing  (being  clean  either  has  or  has  not  brought  satisfaction 
to  oneself,  one's  neighbors,  one's  ancestors).  They  are  often  derived 
and  formulated  to  be  coherent  with  a  wider  body  of  beliefs  and  values. 

And  they  are  readily  applicable,  both  because  of  their  simplicity  and 

because  the  Individual  has  had  practice  In  working  through  their  Implica¬ 
tions  for  various  situations.  Such  facility  should  help  to  guarantee  that 
people  will  give  similar  answers  (regarding,  say,  the  importance  of  clean¬ 
liness),  expressing  the  same  underlying  views,  regardless  of  how  the  ques¬ 
tion  Is  posed. 


The  power  of  these  rules  of  thunb  comes  from  their  development  and  appli¬ 
cation  to  the  settings  found  In  a  simple  and  unchanging  society  with 
repetitive  problems."  Their  viability  becomes  quite  suspect  In  a  world 

These  are,  Incldently,  conditions  quite  similar  to  those  cited  by  Nlsbett 
and  Bellows  (1977)  as  necessary  for  valid  Introspection. 

2 

However,  one  shouldn’t  tout  folk  or  personal  wisdom  too  highly.  Even  In 
those  settings,  people  comfortably  hold  contradictory  adages  ("Nothing 
ventured,  nothing  gained”  and  "Fools  rush  In  where  wise  men  fear  to 
tred").  The  testing  procedures  for  validating  such  wisdom  leaves  much 
to  be  desired.  People  may  not  realize  when  experience  provides  a  test 
for  their  well-worn  rules  and  may  not  remember  their  experiences  pro¬ 
perly  when  they  do  consider  validity.  They  may  forget  a  rule’s  failures 
and  remember  Its  successes  or  vice  versa.  Finally,  the  translation  of 
subjective  feelings  to  observable  judgments  has  an  unavoidable  error 
component  due  to  Inattention,  distraction,  laziness  and  mistakes.  Such 
error  can  Introduce  enough  slippage  Into  the  opinion  evaluation  and 
formulation  process  to  make  clarity  somewhat  difficult. 
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where  the  Issues  are  unfamiliar  and  complex,  the  old  Intuitions  Impotent, 
the  old  rules  untested  and  perhaps  untestable. 

Today  we  are  asked  to  take  responsibility  for  choosing  a  mate,  a  Job,  a 
family  size,  for  guiding  social  policy  and  for  adopting  or  rejecting  new 
technologies.  Each  of  these  Issues  confronts  us  with  greater  freedom  of 
choice  and  more  lasting  consequences  than  ever  before.  They  take  us  Into 
situations  for  which  we  have  never  thought  through  the  Implications  of 
the  values  and  beliefs  acquired  In  simpler  settings.  We  may  be  unfamiliar 
with  the  terms  In  which  Issues  are  formulated  (e.g.,  social  discount  rates, 
miniscule  probabilities,  or  megadeaths).  We  may  have  contradictory  values 
(e.g.,  a  desire  to  avoid  catastrophic  losses  and  a  realization  that  we're 
not  more  moved  by  a  plane  crash  with  500  fatalities  than  by  one  with  300). 
We  may  occupy  different  roles  In  life  (parents,  workers,  children)  which 
produce  clear-cut,  but  Inconsistent  values.  We  may  vacillate  between 
Incompatible,  but  strongly  held,  positions  (e.g.,  freedom  of  speech  Is 
Inviolate,  but  should  be  denied  to  authoritarian  movements).  We  may  not 
even  know  how  to  begin  thinking  about  some  Issues  (e.g.,  the  appropriate 
tradeoff  between  the  opportunity  to  dye  one's  hair  and  a  vague,  minute 
Increase  In  the  probability  of  cancer  20  years  from  now).  Our  views  may 
undergo  predictable  changes  over  time  (say,  as  the  hour  of  decision 
approaches)  and  we  may  not  know  which  view  should  form  the  basis  of  our 
decision.  We  may  see  things  differently  In  theory  than  In  the  flesh. 

We  may  lack  the  mental  capacity  to  think  through  the  Issues  reliably  and 
therefore  come  up  with  different  conclusions  each  time  we  consider  an 
Issue. 

One  possible  partition  of  the  psychological  states  that  might  accompany 
not  knowing  what  we  want  appears  In  Table  1.  Perhaps  the  most  dangerous 
condition  Is  the  first,  having  no  opinion  and  not  realizing  It.  In  that 
state,  we  may  respond  with  the  first  thing  that  comes  to  mind  once  a 
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TABLE  1 


PSYCHOLOGICAL  STATES  ASSOCIATED  WITH  NOT 
KNOWING  WHAT  YOU  WANT 


HAVING  NO  OPINION 


Not  Realizing  It 
Realizing  It 

Living  Without  One 
Trying  to  Form  One 

HAVING  AN  INCOHERENT  OPINION 
Not  Realizing  It 
Realizing  It 

Living  With  Incoherence 

Trying  to  Form  a  Coherent  Opinion 


HAVING  A  COHERENT  OPINION 
Accessing  It  Properly 
Accessing  Only  a  Part  of  It 
Accessing  Something  Else 


question  Is  asked.  As  a  defense  against  uncertainty,  we  may  then  commit 
ourselves  to  maintaining  that  first  expression  and  to  mustering  support 
for  It,  suppressing  other  views  and  uncertainties.  We  may  then  be  stuck 
with  stereotypic  or  associative  responses  reflecting  Inwedlate  stimulus 
configurations  rather  then  serious  contemplation.  Perhaps  the  most  pain¬ 
ful  state  Is  to  acknowledge  having  Incoherent  or  conflicted  values 
requiring  further  analysis. 

The  states  described  In  Table  1  are  determined  In  part  by  the  actual  state 
of  our  values  and  In  part  by  how  we  assess  them  In  a  particular  situation. 
The  critical  elements  of  that  assessment  would  seem  to  be  (a)  our  need 
for  closure.  Itself  a  function  of  the  Importance  of  the  Issue  at  hand, 
the  need  to  act  and  the  audience  for  our  Judgments,  (b)  the  depth  of  the 
analysis,  determined  by  the  thoroughness  of  the  elicitation  procedure  and 
our  general  familiarity  with  the  Issue  at  hand,  and  (c)  our  awareness  of 
the  problems  raised  In  this  paper,  l.e.,  the  possibility  of  not  knowing 
what  we  want  and  the  power  of  the  ellcltor  to  tell  (or  hint)  to  us  what 
our  values  are. 


1.2  Psychophysics  of  Value 

Finding  that  Judgnents  are  Influenced  by  unintended  aspects  of  experimental 
procedure  and  that  those  Influences  are  worthy  of  study  Is  an  oft~told 
tale  In  the  history  of  psychology.  Indeed,  McGuire  (1969b)  describes  much 
of  that  history  as  the  process  by  which  one  scientist's  artifact  becomes 
another's  main  effect.  Central  to  this  process  Is  the  recognition  that 
the  effective  stimulus  cannot  be  presumed  but  must  be  discovered  (Boring, 
1969).  A  selective  survey  of  this  history  appears  In  Table  2.* 


No  attempt  will  be  made  to  docunent  this  Incomplete  list  drawn  from 
various  parts  of  the  lore  of  psychology.  Useful  references  include 
Carterette  »  Friedman  (1974),  Galanter  (1974),  Helson  (1964),  Kllng  & 

Riggs  (1971),  Marks  (1974),  Parduccl  (1974),  Posner  (1973),  Poulton 
(1968),  Rosenthal  &  Rosnow  (1969),  Upshaw  (1974)  and  Woodward  &  Schlosberg 
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TABLE  2 


FROM  ARTIFACT  TO  MAIN  EFFECT 


LABILITY  IN  JUDGMENT  OUE  TO  LED  TO 

ORGANISM 

Inattention,  Laziness,  Fatigue  Repeated  Measures 

Habituation,  Learning,  Matura-  Professional  Subjects 

tlon.  Physiological  Limitations  Stochastic  Response  Models 
Natural  Rhythms,  Experience  With  Psychophysiology 

Related  Tasks  Proactive  and  Retroactive 

Inhibition  Research 


STIMULUS  PRESENTATION 

Homogeneity  of  Alternatives, 
Similarity  of  Successive  Alter¬ 
natives  (Especially  First  and 
Second),  Speed  of  Presentation, 
Amount  of  Information,  Range  of 
Alternatives,  Place  In  Range  of 
First  Alternative,  Olstance 
From  Threshold,  Order  of  Pre¬ 
sentation,  Areal  Extent, 
Ascending  or  Descending  Series 


Classical  Psychophysical  Methods 
The  New  Psychophysics 
Attention  Research 
Range  Frequency  Theory 
Order  Effects  Research 
Regression  Effects 
Anticipation 


TABLE  2  (CONTINUED) 


RESPONSE  MODE 

Stimulus-Response  Compatibility, 
Naturalness  of  Response,  Set, 
Number  of  Categories,  Halo 
Effects,  Anchoring,  Very  Small 
Numbers,  Response  Category 
Labeling,  Use  of  End  Points 


"IRRELEVANT"  CONTEXT  EFFECTS 
Perceptual  Defenses,  Experi¬ 
menter  Cues,  Social  Pressures, 
Presuppositions,  Implicit  Pay¬ 
offs,  Social  Desirability, 
Confusing  Instructions,  Response 
Norms,  Response  Priming,  Stereo¬ 
typic  Responses,  Second-Guessing 


Ergonomics  Research 
Set  Research 
Attitude  Measurement 
Assessment  Techniques 
Contrasts  of  Between  &  Within 
Subject  Design 
Response  Bias  Research 
Use  of  Blank  Trials 

New  Look  In  Perception 
Verbal  Conditioning 
Experimenter  Demand 
Signal  Detection  Theory 
Social  Pressure,  Comparison 
and  Facilitation  Research 


While  no  attempt  has  been  made  at  more  elaborate  categorization  of  these 
variables,  perhaps  the  critical  factor  for  experimental  design  has  been 
whether  an  effect  leads  to  random  or  systematic  variations  In  the  observed 
Jud^nents.  Recognition  of  systematic  effects  Is,  of  course,  most  produc¬ 
tive,  leading  to  the  Identification  of  basic  psychological  principles 
(e.g.,  the  psychological  refractory  period  uncovered  by  varying  speed  of 
stimulus  presentation)  or  theories  (e.g.,  range- frequency  theory  derived 
from  effects  caused  by  varying  the  range  and  homogeneity  of  presented 
stimuli)  or  design  principles  (e.g.,  counterbalancing  for  situations  In 
which  order  effects  have  been  observed).  The  discovery  of  variables  pro¬ 
ducing  random  error  typically  allows  little  response  other  than  estimation 
of  the  size  of  the  effect  and  the  sample  size  needed  to  obtain  desired 
statistical  power.  Although  at  times  noise- reduction  techniques  may  be 
available  (e.g.,  testing  In  the  morning  or  providing  payment  for  accuracy), 
they  are  usually  undertaken  with  some  trepidation  for  fear  of  turning  a 
large  random  error  Into  a  smaller  systematic  one  and  creating  a  task  very 
unrepresentative  of  Its  real-world  analog. 

We  cite  these  effects  for  several  reasons.  One  Is  because  many  of  them 
seem  to  be  as  endemic  to  Judcpents  of  value  as  they  are  to  the  Jud^nental 
context  In  which  they  were  originally  observed.  Parduccl  (1974),  for 
example,  has  found  that  Judged  satisfaction  with  one's  state  in  life  may 
depend  highly  on  the  range  of  states  considered.  According  to  Turner  and 
Krauss  (1978),  order  of  question  presentation  In  surveys  has  had  marked 
effects  on  people's  evaluation  of  the  state  of  the  nation  and  Its  Institu¬ 
tions.  Lichtenstein  and  Slovlc  (1973)  found  that  the  judged  attractiveness 
of  casino  gambles  Is  greatly  affected  by  stimulus-response  compatibility. 
The  second  reason  the  effects  are  cited  Is  to  set  the  stage  for  the 
following  discussion  of  effects  more  specific  to  the  Judgnent  of  values. 
Like  the  phenomena  In  Table  2,  these  effects  may  be  considered  as  today's 
artifacts  on  the  way  to  becoming  tomorrow’s  Independent  variables.  The 


third  reason  Is  to  foreswear  any  pretense  of  trying  to  create  a  scientific 
revolution.  The  pattern  we  are  following  Is  a  hoary  and  respected  one  In 
the  history  of  psychology:  collecting  and  sorting  a  variety  of  documented 
and  suspected  sources  of  lability  In  a  particular  form  of  Judgment.  By 
bringing  together  such  a  diverse  collection  of  effects  we  hope  to  (a) 
facilitate  an  appreciation  of  the  extent  to  which  people's  apparent 
values  are  determined  by  the  ellcltor,  (b)  provide  a  tentative  organiza¬ 
tion  of  effects  and  the  contexts  In  which  they  may  arise,  and  (c)  expli¬ 
cate  the  Implications  of  these  results  for  various  areas  In  basic  and 
applied  psychology. 

1.3  Overview 

If,  as  Rokeach  (1973)  claims,  people  have  relatively  few  basic  values, 
producing  an  answer  to  a  specific  value  question  Is  largely  an  exercise 
In  Inference.  He  must  decide  which  of  our  values  are  relevant  to  that 
situation,  how  they  are  to  be  Interpreted,  and  what  weight  each  Is  to  be 
given.  This  Inferential  process  Is  determined  In  part  by  how  the  question 
Is  defined  and  In  part  by  which  perspectives  we  Invoke  In  solving  the 
Inferential  problem  It  poses.  Once  we  have  reached  a  summary  Judgment, 
we  must  decide  how  strongly  we  believe  In  It  and  In  the  perspectives  upon 
which  It  Is  based. 

As  outlined  In  Table  3,  the  following  three  sections  describe  h<*»  an 
ellcltor  can  affect  the  expression  of  formulation  of  values  by  controlling 
the  definition  of  problems,  the  recruitment  and  Integration  of  perspectives, 
and  the  confidence  placed  In  the  result  of  the  Inferential  process.  That 
control  may  be  overt  or  covert,  deliberate  or  Inadvertent,  reversible  or 
Irreversible.  A  fourth  section  is  devoted  to  the  topic  of  Irreversible 
effects  whereby  the  respondent  Is  actually  changed  by  the  elicitation  pro¬ 
cess,  through  having  existing  perspectives  destroyed  or  new  ones  created. 


TABLE  3 


WAYS  THAT  AN  EUCITOR  HAY  AFFECT 
A  RESPONDENT'S  JUDGMENTS  OF  VALUE 


DEFINING  THE  ISSUE 

Is  There  a  Problem? 

What  Options  and  Consequences  are  Relevant? 

How  Should  Options  and  Consequences  Be  Labeled? 
H<*»  Should  Values  Be  Measured? 

Should  the  Problem  Be  Decomposed? 

CONTROLLING  THE  RESPONDENT'S  PERSPECTIVES 
Altering  the  Salience  of  Perspectives 
Altering  the  Importance  of  Perspectives 
Choosing  the  Time  of  Inquiry 

CHANGING  CONFIDENCE  IN  EXPRESSED  VALUES 
Mlsattrlbutlng  the  Source 
Changing  the  Apparent  Degree  of  Coherence 

CHANGING  THE  RESPONDENT 

Destroying  Existing  Perspectives 
Creating  Perspective 
Deepening  Perspectives 


The  notion  of  an  external  ellcltor  Is  used  mainly  as  a  syntactical  device 
to  avoid  unclear  antecedents.  Questions  of  value  must  be  posed  In  some 
way.  If  an  external  ellcltor  does  not  pose  them  for  us,  then  we  must 
pose  them  for  ourselves  (If  only  by  accepting  some  "natural"  formulation 
offered  by  our  environment).  Indeed,  the  power  of  the  effects  described 
here  may  be  magnified  when  we  pose  problems  to  ourselves,  unless  we 
direct  at  our  own  questions  the  same  critical  eye  that  we  turn  to  some¬ 
one  else  asking  us  about  our  values. 


2.  DEFINING  THE  ISSUE 


2.1  Is  There  a  Problem? 

Before  a  question  of  value  can  be  posed,  someone  must  decide  that  there 
Is  something  to  question.  In  this  fundamental  way,  the  ellcltor  Impinges 
on  the  respondent's  values.  By  asking  about  the  desirability  of  pre¬ 
marital  sex.  Interracial  dating,  dally  prayer,  freedom  of  expression  or 
the  fall  of  capitalism,  the  ellcltor  may  legitimize  events  that  were 
previously  viewed  as  unacceptable  or  cast  doubts  on  events  that  were 
previously  unquestioned.  Opinion  polls  help  set  our  national  agenda  by 
the  questions  they  do  and  do  not  ask.  Advertising  helps  set  our  personal 
agendas  by  the  questions  It  Induces  us  to  ask  ourselves  (two  door  or  four 
door?)  and  those  It  takes  for  granted  (more  Is  better). 

2.2  What  Options  and  Consequences  are  Relevant? 

Once  a  question  has  been  broached.  Its  scope  must  be  specified.  Bounds 
must  be  placed  on  the  options  and  consequences  to  be  considered.  The 
lore  of  survey  research  Is  replete  with  evidence  regarding  the  subtle 
ways  In  which  these  bounds  can  be  controlled  by  the  ellcltor's  demeanor 
and  the  Implicit  assumptions  and  presumptions  In  the  phrasing  of  questions 
(Payne,  1952).  There  are.  It  seems,  many  ways  to  communicate  to  a 
respondent  (a)  whether  the  set  of  possible  options  Is  restricted  to  the 
named,  the  feasible,  the  popular,  or  the  legal,  (b)  whether  new  options 
may  be  created,  and  (c)  whether  the  question  may  be  rejected  out  of  hand. 
The  set  of  relevant  consequences  may  also  be  shaped  to  Include  or  exclude 
Intangible  consequences  (those  without  readily  available  dollar  equiva¬ 
lents),  ethical  (versus  efficiency)  Issues,  social  (versus  personal) 
Impacts,  secondary  and  tertiary  consequences,  means  (versus  ends),  and 
the  well  being  of  nature  (versus  that  of  humans).  Control  may  be 


Inadvertent  as  well  as  deliberate.  For  example,  what  may  seem  to  the 
ellcltor  to  be  Irrelevant  and  dominated  alternatives,  sensibly  deleted 
for  the  sake  of  simplicity,  may  provide  Important  contextual  Information 
for  the  respondent. 

A  tempting  solution  for  the  ellcltor  would  be  to  specify  the  problem  as 
little  as  possible,  leaving  respondents  to  define  the  sets  of  option  and 
consequence  sets  as  they  see  fit.  Unfortunately,  this  approach  Increases 
the  probability  that  the  ellcltor  and  respondent  will  be  talking  about 
different  things  without  solving  the  problem  of  Inadvertent  control. 
Indeed,  one  might  even  argue  that  Impassive  elicitation  Is  the  most  mani¬ 
pulative  of  all.  For  It  means  that  the  entire  questioning  experience  is 
conducted  under  the  Influence  of  the  unanalyzed  predispositions  and  pre- 
sunptlons  of  the  ellcltor  without  even  a  courtesy  warning  to  the  respon¬ 
dent,  "Here  are  my  prejudices,  let's  try  to  be  wary  of  them."  (Rosenthal 
&  Rosnow,  1969).  There  Is  no  reason  to  believe  that  people  will  be 
spontaneously  aware  of  what  has  been  left  out  but  not  brought  to  their 
attention  (Flschhoff,  1977a;  Flschhoff,  Slovlc  A  Lichtenstein,  1978; 
lovlns.  1977;  Nlsbett  A  Wilson,  1977;  Tribe,  Schelllng  A  Voss,  1976). 

2.3  How  Should  Options  and  Consequences  Be  labeled 

The  ellcltor's  Influence  on  the  definition  of  options  and  consequences 
does  not  end  with  their  enumeration.  Once  the  concepts  have  been  evoked, 
they  must  be  given  labels.  As  Harks  (1977)  suggests.  In  a  world  with  few 
hard  evaluative  standards,  such  symbolic  Interpretations  may  be  very 
Important.  While  the  facts  of  abortion  remain  constant.  Individuals  may 
vacillate  In  their  attitude  as  they  attach  and  detach  the  label  of 
“murder."  The  value  of  a  dollar  may  change  greatly  If  It  Is  called 
"discretionary  funds,"  "public  funds,"  or  "widows'  and  orphans'  funds." 
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Political  scientists  have  been  accused  of  Ideologically  biasing  their 
research  by  describing  acts,  options  and  outcomes  with  terms  drawn  from 
neo-classical  economics  with  Its  particular  (mostly  conservative) 
political  bias  (Ashcraft,  1977).  More  generally,  Karl  Mannheim  (1936) 
observed  that  "the  political  theorist's. ..most  general  mode  of  thought 
Including  even  his  categories  Is  bound  up  with  general  political  and 
social  undercurrents. . .extending]  even  Into  the  realm  of  logic  Itself" 

(p.  117).  Presunably,  political  scientists'  choice  of  language  Imposes 
that  perspective  on  respondents  to  their  surveys  and  readers  of  their 
texts. 

While  not  new,  these  Issues  are  still  troublesome.  Furthermore,  they 
cannot  be  avoided,  for  some  meaning  must  be  given  to  events,  and  the 
meaning  generated  by  the  respondent  may  be  even  less  appropriate  than 
that  Imposed  by  the  ellcltor  (Poulton,  1977).  When  the  respondent  sees 
the  validity  of  contradictory  symbolic  meanings  (e.g.  ,  abortion  both  Is 
and  Is  not  murder),  conflict  In  meaning  cannot  be  resolved.  In  such 
cases,  the  only  recourse  Is  to  step  back,  somehow,  and  decide  on  exogenous 
grounds  just  what  this  elicitation  session  Is  all  about.  If  necessary, 
that  longer  look  should  come  sooner  rather  than  later.  Often,  changes  In 
perspective  are  Irreversible  (Fischhoff,  1977b).  The  psychological 
impact  of  an  offered  Interpretation  may  not  be  resclndable  (try  to  forget 
that  "this  Is  what  I,  your  mother,  want  you  to  do,  but  decide  for  your¬ 
self"  or  that  "this  Is  your  childhood  sweetheart’s  favorite  restaurant"). 

2.4  How  Should  Values  Be  Measured? 

After  the  problem  has  been  structured,  the  units  of  measurement  must  be 
chosen.  It  Is  not  difficult  to  construct  options  whose  relative  desira¬ 
bility  Is  changed  when  the  evaluation  criterion  undergoes  any  of  the 
following  shifts:  (a)  from  profit  to  regret,  (b)  from  maximizing  to 


satisficing:  (c)  from  the  fair  price  to  the  price  I'd  pay,  (d)  from 
final  asset  position  to  changes  In  asset  position,  (e)  from  the  price 
I'd  pay  to  avoid  a  malady  to  the  price  I'd  have  to  be  paid  to  accept  It, 

(f)  from  lives  saved  to  lives  lost,  and  (g)  from  the  ratio  of  benefits 

4 

to  costs  to  the  difference  between  benefits  and  costs.  As  before, 
choice  of  units  may  be  specified  by  the  ellcltor  or  left  to  that  nether 
region  created  by  the  “neutral''  stance  of  non-specification. 

Moreover,  the  site  of  the  unit  chosen  may  affect  the  responses.  Unless 
some  help  Is  provided  to  the  respondent  (say,  through  the  use  of  anchors 
or  logarithmic  scales).  It  may  be  very  difficult  to  express  values  that 
range  over  several  orders  of  magnitude  for  a  given  set  of  stimuli  because 
people  find  It  hard  to  use  either  very  small  or  very  large  nunbers 
(Poulton,  1968). 

2.5  Should  The  Problem  Be  Decomposed? 

Many  (or  most)  Interesting  questions  of  value  are  subtle,  complex  and 
multifaceted,  with  Intricate  Interrelations  and  consequences.  The 
ellcltor  must  choose  between  presenting  the  event  to  be  evaluated  as  a 
whole  or  offering  some  kind  of  decomposition.  Offering  an  unanalyzed 
whole  Incurs  the  risk  that  the  respondent  will  latch  on  to  a  single 
aspect  of  the  problem  or  treat  all  aspects  superficially,  so  as  to  mini¬ 
mize  cognitive  strain. 

Unfortunately,  the  act  of  decomposition  has  consequences  besides  clarifi¬ 
cation.  One  charge  leveled  against  dlvlde-and-conquer  strategies  Is  that 

^Kahneman  and  Tversky  (In  press)  provide  the  most  extensive  and  Insightful 
discussion  of  the  power  of  shifts  In  point  of  reference,  the  principle 
underlying  many  of  these  effects. 
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they  destroy  the  Intuitions  of  the  respondent  (Dreyfus  4  Dreyfus,  n.d.). 
Drawing  on  the  work  of  Gestalt  psychologists  and  Polanyl  (e.g.,  1962), 
these  critics  argue  that  people  think  most  naturally  and  adequately  by 
analogy  with  past  experiences  and  that  all  such  thought  (regarding  Issues 
of  fact  or  value)  Is  context  dependent.  Therefore,  any  attempt  to  evalu¬ 
ate  separately  the  attributes  of  a  particular  event  or  designate  the 
Importance  of  attributes  In  the  abstract  Is  likely  to  produce  spurious 
results.  In  addition  to  destroying  the  respondent's  natural  understanding, 
decomposition  procedures  may  Impose  a  response  mode  that  does  not  allow 
people  to  articulate  their  understanding  of  (holistic)  value  Issues. 5 

Furthermore,  decompositions  are  not  unique;  different  cuts  may  lead  to 
different  judgments  of  the  same  Issue.  Sequential  evaluation  of  alter¬ 
natives  has  been  found  to  produce  different  preferences  than  simultaneous 
evaluation  (Tversky,  1969).  Plott  and  Levine  (1978)  have  shown  that  the 
order  In  which  attributes  are  considered  Is  a  crucial  variable  In  deter¬ 
mining  preference  orderings.  Some  theories  of  choice  (Aschenbrenner, 

1978)  predict  shifts  In  the  attractiveness  of  simple  gambles  as  a  function 
of  their  decomposition.  Kahneman  and  Tversky  (In  press)  demonstrated  a 
variety  of  reversals  In  preference  depending  on  whether  prospects  were 
considered  as  a  whole  or  decomposed  Into  two  stages.  The  effective  ele¬ 
ment  here  was  Isolating  (In  the  first  stage)  one  sub-outcome  that  was 


If  true,  this  criticism  would  attribute  the  greatest  validity  to  elicita¬ 
tion  procedures  that  leave  options  In  their  most  natural  form.  For 
example,  Hanroond's  social  judgment  theory  approach  (Hamnond  4  Adelman, 
1976)  In  which  complete  options  are  Judged  should  be  preferred  to  the 
Keeney  and  Ralffa  (1976)  procedure  In  which  whole  options  are  evaluated 
but  only  two  attributes  are  varied  at  a  time.  That  procedure.  In  turn, 
should  be  preferred  to  Edwards'  (Gardiner  4  Edwards,  1975)  SMART  method 
that  forces  total  decomposition.  Ironically,  Dreyfus  and  Dreyfus  (n.d.) 
chose  Hanmond  and  Adelman  (1976)  as  a  case  In  point  for  the  flaws  of 
decomposition. 


known  for  certain.  Certain  losses  and  gains  are  weighted  more  heavily 
than  uncertain  outcomes  In  determining  overall  attractiveness. 


Finally,  as  Tribe  has  argued  (1972),  decomposition  Itself  typically 
carries  a  message.  It  stresses  ends  over  means.  It  proclaims  the 
superiority  of  the  el  Id  tor’s  overall  perspective  (and  the  overall  social 
Importance  of  analysis  and  Its  purveyors,  Gouldner,  1976).  It  conveys  a 
message  of  analyzablllty  or  solvability  where  that  may  be  Inappropriate. 
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3.  CONTROLLING  THE  RESPONDENT'S  PERSEPECTIVE 


3.1  Altering  the  Salience  of  Perspectives 

People  solve  problems.  Including  the  determination  of  their  own  values, 
with  what  comes  to  mind.  The  more  detailed,  exacting  and  creative  their 
Inferential  process  Is,  the  more  likely  they  are  to  think  of  all  they 
know  about  a  question.  The  briefer  that  process  becomes,  the  more  they 
will  be  controlled  by  the  relative  accessibility  of  various  considerations. 
Accessibility  may  be  related  to  Importance,  but  It  Is  also  related  to 
degree  of  associative  priming,  the  order  In  which  questions  are  posed, 
Imaginablllty,  concreteness  and  other  factors  only  loosely  related  to 
Importance. 

One  way  In  which  the  ellcitor  may  unintentionally  prime  particular  consi¬ 
derations  Is  seen  In  Turner  and  Krauss'  (1978)  observation  that  people's 
confidence  In  national  Institutions  was  substantially  higher  In  a 
National  Opinion  Research  Center  poll  than  In  a  Harris  poll  taken  at  the 
same  time  when  the  latter  prefaced  the  confidence  questions  with  six 
items  relating  to  "political  alienation."  Another  Is  Flschhoff,  Slovlc, 
Lichtenstein,  Read  and  Combs'  (1968)  finding  that  people  judged  the  risks 
associated  with  various  technologies  to  be  more  acceptable  following  a 
Judgnent  task  concerning  the  benefits  of  those  technologies  than  follow¬ 
ing  a  task  dwelling  on  their  risks.  According  to  Wlldavsky  (1966),  the 
very  act  of  asking  people  for  their  own  personal  values  may  suppress  the 
availability  of  social  values.  Indeed,  one  could  speculate  that,  in 
general,  when  conflicting  values  are  relevant  to  a  particular  Issue, 
the  priming  or  evocation  of  one  will  tend  to  suppress  the  accessibility 
of  its  counterpart. 


Expressed  values  sometimes  reflect  the  direct  application  of  established 
rules.  Consistency  with  past  preferences  Is  one  such  rule;  cautiousness 
Is  another.  Whether  or  not  a  rule  Is  evoked  will  depend  upon  situational 
cues.  As  an  example  of  a  rule  that  needed  to  be  evoked  before  It  was 
used,  we  have  found  that  most  people  will  prefer  a  gamble  with  a  .25 
chance  to  lose  $200  (and  a  .75  chance  to  lose  nothing)  to  a  sure  loss 
of  $50.  However,  when  that  sure  loss  Is  called  an  Insurance  prcmlvm. 
people  will  reverse  their  preferences  and  forego  the  $50.  For  these 
people.  Insurance  was  an  acceptable  but  Initially  Inaccessible  rule; 
without  a  specific  prompt,  the  sure  loss  was  not  seer  as  a  premlun. 

3.2  Altering  the  Importance  of  Perspectives 

Once  an  ensemble  of  relevant  values  has  been  elicited,  some  order  must 
be  placed  on  them.  This  ordering  or  weighting  may  also  fall  under  ellcl- 
tor  control.  Such  control  Is,  In  fact,  what  experimental  demand  charac¬ 
teristics  are  all  about:  unintentionally  telling  the  subject  how  to 
think,  what  to  look  at  and  wnat  Is  expected.  The  unintended  Impacts  of 
ellcltor  expectancies  show  the  power  of  Inadvertent  Influence  (Rosenthal, 
1969).  Although  Rosenthal  minimizes  the  Importance  of  operant  condition¬ 
ing  In  such  Influence,  It  Is  not  hard  to  Imagine  the  Impact  of  an  Incre¬ 
dulous  "hnm"  or  a  querulous  "half  as  Important?"  on  the  behavior  of  a  con 
fused  or  uncertain  respondent.  Nor  Is  It  hard  to  Imagine  how  the  demeanor 
of  the  ellcltor  might  encourage  or  discourage  the  weight  given  to  Intan¬ 
gible  or  non-western  values.  Canavan-Gumpert  (1977)  has  shown  how  reward 
and  critics*  can  shift  people's  attention  between  the  costs  and  benefits 
Involved  with  a  particular  event. 

One  unavoidable  decision  made  by  the  ellcltor  that  may  have  great  Influ¬ 
ence  on  the  values  that  emerge  Is  choice  of  response  mode.  Lichtenstein 
and  Slovlc  (1971,  1973;  see  also  llndman,  1971,  and  Grether  &  Plott,  In 


3-2 


press)  showed  that  people  use  different  cognitive  processes  when  evaluating 
the  worth  of  gambles  via  a  comparative  mode  ("Which  would  you  rather  play?”) 
than  they  use  when  judging  each  gamble  separately  (“How  much  Is  playing 
each  worth  to  you?").  The  different  processes  triggered  by  the  change  In 
response  mode  lead  people  to  rather  awkward  reversals  of  preference  (“I 
prefer  A,  but  attach  a  higher  value  to  B“).  One  possible  explanation  of 
such  reversals,  based  on  related  work  by  Tversky  (1972)  and  Slovlc  (1975), 

Is  that  people  make  choices  by  searching  for  rules  or  concepts  that  provide 
a  good  justification,  that  minimize  the  lingering  doubts,  and  that  can  be 
defended  no  matter  what  outcome  occurs  (example:  "Quality  Is  more  impor¬ 
tant  than  quantity").  Different  response  modes  Increase  the  Importance 
of  different  rules.  In  the  gambles  example,  A  offered  a  higher  probability 
of  winning  while  B  promised  a  greater  payoff.  Here  the  preference  mode 
may  have  emphasized  that  "the  stakes  don't  matter  If  you're  not  going  to 
win  anyway,"  while  the  bidding  mode  focused  attention  on  the  payoff. 

Another  effect  peculiar  to  choice  behavior  was  found  by  Slovlc  and  Mac- 
Phi  llamy  (1974),  who  observed  that  dimensions  common  to  each  alternative 
had  greater  Influence  on  choices  than  did  dimensions  that  were  unique  to 
a  particular  alternative.  Interrogation  of  the  respondents  after  the 
study  Indicated  that  most  did  not  Intend  to  give  more  weight  to  tne  common 
dimension  and  were  unaware  that  they  had  done  so. 

3.3  Choosing  the  Time  of  Inquiry 


People’s  values  change  over  time,  sometimes  systematical ly,  sometimes 
not.  The  point  In  time  at  which  the  ellcltor  chooses  to  Impinge  on  the 
respondent  will  determine  In  part  what  the  respondent  says.  Some  changes 
are  secular  and  relatively  Irreversible.  A  society  and  Its  members  may 
become  more  or  less  predisposed  to  consider  environmental  values  (Harblln, 
1977)  or  equity  Issues  or  the  rights  of  women  as  time  goes  on.  The  age 


distribution  In  that  society  as  a  whole  nay  be  shifting,  leading  to  a 
greater  preponderance  of  young  or  old  people  with  their  characteristic 
perspectives.  By  waiting  or  by  hastening,  an  ellcltor  has  some  power  to 
create  a  different  picture  of  people's  expressed  values. 

Other  changes  over  tine,  with  varying  degrees  of  predictability,  are 
naturatlon,  satiation,  cumulative  deprivation.  Increasing  risk  aversion 
as  one  approaches  an  event,  mood  changes  with  time  of  day,  day  of  the 
week,  or  season  of  the  year.  Consider  people  who  regularly  take  stock 
of  the  world  late  at  night  and  whose  existential  decisions  are  colored 
by  their  depleted  body  state.  Is  that  value  to  be  trusted  or  should  one 
rely  on  the  way  they  value  their  lives  at  high  noon  on  a  bright  spring 
day?  Should  an  ellcltor  rely  on  an  auto  worker's  opinion  of  the  intrinsic 
satisfaction  of  assembly-line  work  on  the  bus  Monday  morning  or  while  on 
holiday  and  refreshed?  In  a  mgltlple-play  experiment  on  Insurance-buying 
behavior  (Slovlc,  Flschhoff,  Lichtenstein,  Corrigan  &  Combs,  1977),  we 
found  that  participants  who  were  generally  risk  seeking  shifted  to  risk 
aversion  on  the  final  round  (just  before  cashing  out).  Which  attitude 
should  we  say  characterized  them?  Or  might  not  both  of  these  perspec¬ 
tives  be  part  of  the  Individual's  value  system? 

Any  gap  In  time  between  Judgnent  of  an  event  and  Its  occurrence  may 
Introduce  an  element  of  random  or  systematic  variation  In  people's  Judg¬ 
ment.  Hypotfetlcal  Judgments  of  what  an  event  would  be  like  may  not 
capture  how  It  will  look  In  the  flesh.  The  contrast  between  the  limited 
funds  budgeted  for  rescue  operations  and  disaster  relief  and  the  almost 
unlimited  resources  made  available  for  a  particular  rescue  Is  one  pro¬ 
duct  of  this  failure  of  anticipation,  as  Is  our  greater  readiness  to 
pay  for  the  protection  of  known  rather  than  statistical  lives  (Fried, 
1969).  We  know  relatively  little  about  people’s  ability  to  anticipate 
the  Impact  that  specified  future  contingencies  will  have  on  their 


( 
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perceptions  and  values--nor  which  perspective,  the  anticipated  or  the 
actual.  Is  a  better  guide  to  action  (or  true  preferences).  The  scanty 
evidence  we  have  suggests  that  sometimes  at  least  It  Is  better  to  go 
with  one’s  anticipations  If  derived  In  a  relatively  thoughtful  setting 
(Flschhoff,  1978). 


4.  CHANGING  CONFIDENCE  IN  EXPRESSED  VALUES 


The  power  of  values  comes  from  their  roles  as  guides  to  actions,  as 
embodiments  of  ourselves,  and  as  expressions  of  our  relation  to  the 
world  (Rokeach,  1973).  It  may  matter  greatly  what  we  think  their  source 
to  be,  how  strongly  we  believe  In  them,  and  how  coherent  they  seem. 
Attitudes  towards  values  may,  however,  be  as  labile  as  the  values  them¬ 
selves. 


4. 1  Mlsattrlbutlnq  the  Source 

Much  of  the  history  of  social  psychology  Involves  attempts  to  get  people 
to  mlsattrlbute  the  source  of  their  values,  by  counter-attl tudlnal  role- 
playing,  by  exposure  to  undirected  (overheard)  conversations,  by  con¬ 
formity  pressure,  or  by  Inducing  social  comparison  processes.  These  mani¬ 
pulations  lead  people  to  adopt  as  their  own,  without  critical  analysis, 
attitudes  that  originated  with  others  (McGuire,  1969a).  Cognitive 
psychology  offers  some  new  wrinkles  In  this  mlsattrlbutlon  process, 
showing  the  ease  with  which  presuppositions  are  absorbed  as  facts 
(loftus.  Miller  &  Burns,  1978),  Inferences  are  confused  with  direct  ob¬ 
servations  (Harris  4  Monaco,  1970),  mere  repetition  Improves  the  believa- 
blllty  of  statements  (Hasher,  Goldstein  4  Topplno,  1977)  and  people  ego- 
centrical  ly  assune  that  others  share  their  views  (Ross,  Greene  4  House, 
1977). 


4.2  Changing  the  Apparent  Degree  of  Coherence 

People  will  act  and  press  others  to  act  on  values  In  which  they  believe 
most  deeply.  Depth  of  belief  Is  a  function  of  source,  as  mentioned,  and 
of  the  degree  to  which  such  values  appear  to  be  In  conflict.  A  super¬ 
ficial  analysis  may  create  an  Illusion  of  confidence  In  values  simply 


because  conflicting  values  are  not  considered.  Incoherence  In  beliefs 
Is  typically  apparent  only  when  the  ellcltor  adopts  or  encourages 
different  perspectives.  It  Is  easy  to  avoid  taking  that  extra  step, 
particularly  when  the  respondent  Is  Interested  In  keeping  things  simple. 

Such  collusion  toward  simplicity  Is  encouraged  by  one  Implicit  message  of 
many  elicitation  procedures:  "This  topic  Is  knowable,  analyzable;  after 
one  session,  we  will  both  know  your  values."  It  Is  magnified  by  the  aura 
of  precision  and  professionalism  fostered  by  elaborate,  nunerlcal  response 
modes.  That  aura  manifests  the  can-do  technological-fix,  mastery-of-the- 
world  attitude  that  characterizes  our  society  (Tribe  et  al.,  1976). 

Ellul  (1969)  has  argued  that  one  way  to  control  people's  minds  Is  to 
lead  them  to  believe  that  they  can  have  an  opinion  on  anything  and  every¬ 
thing.  Those  opinions  will  necessarily  be  superficial,  guaranteeing  that 
people  will  have  elaborated,  thoughtful  positions  on  nothing.  When  we 
ask  or  answer  questions  of  value  a  useful  antidote  to  overconfidence 
might  be  to  recall  the  effort  Invested  by  Rawls  (1971)  and  his  colleagues 
to  produce  a  reasonably  coherent  position  on  Just  one  difficult  value 
Issue,  social  justice. 

It  Is,  of  course,  natural  to  feel  that  we  are  the  ranking  experts,  the 
final  arbiters  of  our  own  values.  Yet  In  order  to  know  how  good  our 
best  assessment  of  those  values  Is,  we  must  recognize  the  extent  to  which 
they  are  under  the  control  of  factors  that  we  (as  scientists  as  well  as 
Individuals)  understand  rather  poorly. 
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S.  CHANGING  THE  RESPONDENT 


In  most  of  the  effects  cited  above,  the  el  Id  tor  neither  creates  nor 
destroys  values,  but  merely  affects  the  ways  In  which  they  are  accessed, 
organized  and  evaluated.  Some  effects,  however,  suggest  ways  In  which 
the  respondent  may  be  Irreversibly  changed  by  the  questioning  procedure, 
perhaps  for  the  better,  perhaps  for  the  worse.  These  fall  Into  three 
generic  categories.  The  ellcltor  may  destroy  an  existing  perspective  on 
a  value  Issue,  create  a  perspective  where  none  existed  before  or  deepen 
the  respondent’s  understanding  of  the  Issue  at  hand  or  of  value  questions 
In  general. 

5.1  Destroying  Existing  Perspectives 

As  mentioned,  one  charge  leveled  against  those  who  break  complex  ques¬ 
tions  of  value  Into  more  manageable  component  questions  Is  that  their 
dlvlde-and-conquer  strategy  destroys  the  Intuitions  of  their  respondent. 

A  generalization  of  this  position  might  be  that  any  elicitation  procedure 
deviating  from  the  normal  way  In  which  Judgments  are  made  may  erode  the 
respondent's  ■feel'*  for  the  Issue  at  hand.  The  failure  of  formal  decision¬ 
making  procedures  to  attract  the  loyalty  of  corporate  decision  makers  has 
repeatedly  been  attributed  to  these  Individuals'  refusal  to  trade  the  com¬ 
forts  of  their  Intuitions  for  the  promises  of  the  formal  methods  (Harrison 
1977). 

Other  aspects  of  an  elicitation  mode  may  destroy  parts  of  our  "natural" 
perspective  on  Issues  (Barnes,  1976).  For  example,  the  dyadic  nature  of 
the  elicitation  procedure,  with  an  ellcltor  who  Is  reluctant  to  Influence 
the  response,  may  deprive  the  respondent  of  the  opportunity  to  Invoke 
social  comparison  processes  (Upshaw,  1974).  Discussion  with  others  may 
be  a  natural  part  of  the  way  In  which  many  people  formulate  their  judg¬ 
ments.  It  may  also  be  an  effective  procedure,  perhaps  by  recruiting 


additional  Information  and  externalizing  alternative  perspectives  that 
are  too  difficult  to  carry  In  one’s  head  simultaneously.  In  these 
examples,  the  elicitation  procedure  may  be  seen  as  destroying  respondents' 
natural  perspective  by  depriving  them  of  tools  upon  which  they  are 
accustomed  to  rely  (Edwards,  1975). 

5.2  Creating  Perspectives 

An  Insidious  possibility  when  posing  unfamiliar  questions  to  Individuals 
with  poorly  formulated  opinions  Is  covertly  creating  a  perspective  where 
none  existed.  One  possible  process  for  accomplishing  this  feat  Is  for 
the  respondent  to  satisfy  the  ellcltor's  hunger  for  a  recordable  response 
by  saying  whatever  comes  to  mind.  Once  emitted,  this  associative 
response  may  assune  a  life  of  Its  own.  The  respondent  may  subsequently 
conclude  "If  that's  what  I  said,  then  that  must  be  what  I  meant"  (Bern, 
1972).  As  shorn  In  studies  of  counter-attltudlnal  role  playing  (McGuire, 
1969a),  such  positions  can  show  a  tenacity  which  Is  Independent  of  their 
source  or  validity  (Ross,  1977).  The  fact  that  such  spontaneous  responses 
are  provided  In  a  formal  setting  with  a  relatively  esteemed  listener  may 
heighten  such  commitment  effects,  leading  to  newly  Invented  but  firmly- 
held  values.  The  very  fact  that  one  Is  out  of  one's  depths  In  such 
situations  makes  It  quite  difficult  to  get  a  critical  view  on  this  new 
perspective. 

Elicitation  may  Induce  people  to  think  about  Issues  they  wish  to  avoid 
and  would  have  Ignored  had  they  not  been  "bullied"  by  the  ellcltor.  In 
some  cases,  the  ellcltor  cannot  be  faulted  for  forcing  people  to  take  their 
heads  out  of  the  sand  and  face  the  Issues  Implicit  In  the  decision  they 
must  make  In  any  case.  The  use  of  decision  analysis  In  medical  contexts 
will  create  many  such  situations  as  physicians  and  patients  are  forced 
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to  provide  explicit  values  for  pain  and  death  (Bunker,  Barnes  &  Hosteller, 
1977).  In  other  cases,  the  ellcltor  may  be  asking  respondents  to  abrogate 
their  own  rights  by  telling, say,  how  much  they  would  have  to  be  compen¬ 
sated  for  a  particular  degradation  to  their  environment  without  offering 
the  response  option  "a  clean  environment  Is  non-negotlable"  (Brookshire, 
Ives  &  Schulze,  1976).  In  the  extreme,  the  ellcltor  may  be  guilty  of 
"anaesthetizing  moral  feeling"  by  Inducing  the  respondent  to  think  about 
the  unthinkable  (Tribe,  1972).  The  mere  act  of  thinking  about  some 
Issues  In  "cold,  rational"  terms  may  lead  to  the  legitimization  of  alter¬ 
natives  that  should  be  dismissed  outright. 

5.3  Deepening  Perspectives 

While  the  preceding  discussion  has  emphasized  unsavory  aspects  of  the 
impact  of  the  ellcltor  on  the  respondent,  there  obviously  are  situations 
In  which  the  only  valid  elicitation  procedure  Is  a  reactive  one.  Consi¬ 
der  a  national  poll  of  values  on  Issues  relevant  to  nuclear  waste  disposal, 
the  results  of  which  will  be  used  to  guide  policy  makers.  An  Individual 
who  has  no  elaborated  beliefs  may  not  be  responding  In  his  or  her  best 
Interests  by  giving  the  value  the  question  seems  to  hint  at.  On  the 
other  hand,  providing  no  response  effectively  constitutes  disenfranchise¬ 
ment.  An  ellcltor  might  reasonably  be  expected  to  help  In  translating 
the  respondent's  basic  predispositions  Into  codable  judgments  whose  Impli¬ 
cations  and  assumptions  are  well  understood.  Surely  an  ellcltor  does 
small  service  to  a  respondent  with  Incoherent  values  by  asking  questions 
that  tap  only  a  part  of  those  values,  particularly  If  that  part  might 
be  abandoned  (or  endorsed  more  heartily)  upon  further  contemplation. 


How  might  the  ellcltor  deepen  the  respondent's  perspective  without  unduly 
manipulating  It?  One  reasonably  safe  way  may  be  to  help  the  respondent 


work  through  the  logical  Implications  of  various  points  of  view.  We 
presented  college  students  and  members  of  the  League  of  Women  Voters  with 
the  two  tasks  In  Figure  1.  The  first  asked  them  to  choose  between  a 
high-variance  and  a  low-variance  option  Involving  the  loss  of  life.  The 
second  asked  them  to  choose  one  of  three  functions  as  representing  the 
way  In  which  society  should  evaluate  lives  In  multi-fatality  situations. 
Its  Instructions  (omitted  In  Figure  1)  provided  elaborate  rationales  for 
adopting  each  of  the  three  function  forms.  The  predominant  response 
pattern,  chosen  by  over  half  of  all  subjects,  was  Option  A  In  the  civil 
defense  question  and  Curve  2  In  the  second  task.6  The  former  Indicates 
a  risk-seeking  attitude  toward  the  loss  of  life.  The  latter  Indicates  a 
risk-averse  attitude.  Confronting  subjects  with  this  Inconsistency 
allowed  them  the  opportunity  to  reflect  on  Its  source  and  on  their  true 
values. 

Many  social  decisions  require  people  to  determine  desirable  rates  for 
growth  or  for  discounting  future  costs  and  benefits.  Wagenaar  and 
Sagarla  (1975)  have  shown  that  people  have  very  poor  Intuitions  on  the 
cumulative  Impact  of  those  rates  when  they  are  compounded  over  a  period 
of  years.  "Neither  special  Instructions  about  the  nature  of  exponential 
growth  nor  dally  experience  with  growth  processes  enhanced  the  extrapola¬ 
tions"  (p.  416).  When  Issues  with  compounded  rates  arise,  the  ellcltor 


These  results  were  not  changed  appreciably  either  by  changing  the  degree 
of  elaboration  In  the  rationale  given  for  the  three  curves,  nor  by  des¬ 
cribing  civil  defense  Option  B  as  an  action  option  that  reduced  the 
nunber  of  casualties  (to  a  small,  but  definite,  number).  The  civil 
defense  question  was  posed  In  nine  ways,  varying  the  variance,  expecta¬ 
tion  and  probability  of  loss  with  Option  A  (with  B  always  a  sure  loss 
of  A's  expectation).  Option  B  was  never  chosen  by  more  than  10*  of 
subjects  except  In  the  one  case  where  A  specified  a  .99  chance  of  losing 
no  lives  and  a  .01  chance  of  losing  100  lives,  while  B  specified  the 
certainty  of  losing  one  life. 


5-4 


should  work  through  the  details  of  the  extrapolations,  leaving  nothing 
to  the  Imagination. 

A  more  difficult  Intervention  Is  to  educate  respondents  about  the  assump¬ 
tions  upon  which  their  beliefs  are  contingent.  Tougher  still  Is  trying 
to  comnunlcate  factual  Information  the  respondent  may  not  have  known  or 
taken  Into  consideration.  Kunreuther  et  al.  (1978)  found  that  residents 
of  hatard-prone  areas  typically  underestimate  the  likely  property  damage 
from  floods  and  overestimate  that  to  be  expected  from  earthquakes. 
Although  there  are  obvious  problems  with  presenting  damage  Information 
without  unfairly  Influencing  subsequent  judgnents,  It  would  seem  to  be  a 
valid  Input  to  helping  someone  evaluate  the  national  flood  Insurance 
program.  Likewise,  Just  telling  people  In  vivid  detail  what  they  may 
experience  In  a  new  job  can  increase  their  probability  of  success  and 
satisfaction  (Mitchell  &  Beach,  1977). 
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6.  IMPLICATIONS  FOR  RESPONDENTS 


Hon  do  we  manage  to  get  by  with  so  much  Incoherence  In  our  beliefs?  Why 
are  we  not  paralyzed  with  Indecision  (to  the  extent  that  we  are  aware  of 
that  Incoherence)  or  punished  by  the  consequences  of  acting  on  conflict¬ 
ing  views? 

Paralysis  seems  averted  by  the  non-lntultlve  nature  of  the  effects  des¬ 
cribed  here  and  the  fact  that  the  world  seldom  asks  us  more  than  one 
question  on  a  given  topic.  If  we  are  confronted  with  Inconsistency,  It 
Is  relatively  easy  to  define  our  way  out  of  contradictions  with  specious 
argunents  like  "that’s  different,"  "things  have  changed"  or  "It  all 
depends."  There  Is  always  some  extraneous  factor  that  can  be  Invoked  to 
explain  a  difference.  According  to  Rokeach  (1973),  people  experience 
discomfort  at  Inconsistency  In  their  values  only  when  It  hints  at  Incom¬ 
petence  or  immorality.  For  better  or  worse  we  are  usually  spared  that 
experience.  Table  4  lists  some  ways  one  might  deal  with  Incoherence. 

An  Intriguing  option  Is  Just  living  with  Incoherence.  In  the  experiment 
described  In  Figure  1,  half  of  the  subjects  had  Inconsistent  preferences. 
Of  those,  half  decided  to  deny  the  Incoherence;  most  of  these  offered  no 
argunent  at  all,  although  some  tried  to  demonstrate  an  underlying 
coherence  by  a  deeper  analysis  of  their  own  preferences  (typically,  by 
specifying  domains  In  which  risk  seeking  and  risk  aversion  were  appro¬ 
priate).  A  more  satisfying  solution  Is  to  think  one’s  way  through  to 
coherence.  Such  analytic  resolution  might  Involve  devising  new,  con¬ 
flict-free  options  or  recognizing  that  the  problem  at  hand  Is  misstated. 

We  may  escape  punishment  for  acting  on  Incoherent  values  because  (a) 
day-to-day  life  affords  us  much  opportunity  for  hands-on  experience 
that  obviates  the  need  for  analytic  judgnent;  (b)  we  are  proficient  at 


convincing  ourselves  that  we  like  what  we  get  (sour  grapes,  dissonance 
reduction)  and  (c)  we  cannot  easily  evaluate  the  outcomes  of  our 
decisions  (tinhorn  &  Hogarth,  1978;  Flschhoff,  In  press).  Unbeknownst 
to  ourselves,  we  may  be  stunbllng  all  the  time,  tripped  up  by  our  own 
Inconsistent  values.  The  chaos  reigning  In  our  society's  attempts  to 
regulate  various  technological  hazards  suggests  a  lot  of  counterproduc¬ 
tive  effort  (Kates,  1977;  lowrance,  1976). 


7.  implications  for  elicitors 


The  purveyors  of  formal  methods  of  decision  making  constitute  one  group 
of  elicitors.  Decision  analysts  (and  economists  and  operations 
researchers)  not  only  elicit  values,  but  take  the  nunbers  they  receive 
seriously  In  determining  decisions  that  are  (purportedly)  In  the  respon¬ 
dents'  best  Interests.  The  possibility  of  Instability  In  values  Is 
typically  treated  by  sensitivity  analysis.  The  analyst  recalculates 
the  decision  model  while  allowing  one  value  at  a  time  to  vary  over  Its 
range.  If  the  final  recomnendatlon  Is  Insensitive  to  changes  In  each 
value  variable,  then  the  Instability  Is  considered  to  be  Inconsequential. 

Although  we  have  only  the  rudiments  of  a  theory  describing  the  effect 
of  Instability  on  decisions  (Fischer,  1976;  Flschhoff,  In  press),  some 
preliminary  results  suggest  that  the  expected  value  of  continuous 
decisions  (e.g..  Invest  X  dollars)  Is  relatively  Insensitive  to  shifts 
In  Individual  values.  Thus,  one  dose  of  one  of  the  psychophysical 
effects  described  In  Table  2  might  not  have  too  much  Impact.  Unfortu¬ 
nately,  little  Is  known  about  how  multiple  errors  compound  within  an 
analysis,  nor  what  Is  the  effect  of  correlated  errors.  The  use  of  one 
perspective  throughout  an  analysis  (the  usual  practice)  may  produce  many 
shifts  of  response  In  the  same  direction.  For  example,  one  might  per¬ 
sistently  deflate  the  apparent  Importance  of  environmental  values  or 
reduce  the  dlscrlmlnablllty  of  values  of  all  sorts. 

Whatever  the  promise  of  sensitivity  analysis.  In  some  contexts  It  com¬ 
pletely  misses  the  point.  Many  of  the  effects  described  here  reflect 
the  Introduction  of  distorted  perspectives  or  newly  created,  possibly 
foreign,  values  Into  a  decision-making  process.  Blanket  Invocation  of 
sensitivity  analysis  will  not  excuse  the  Imposition  of  an  ellcltor's 
perspective  on  the  respondent.  When  shifts  In  perspective  lead  to 


reversals  of  preference,  sensitivity  analysis  avoids  the  real  Issue  of 
which  perspective  Is,  In  fact,  appropriate.  Furthermore,  the  long-range 
goal  of  Involving  people  In  decision  making  should  be.  In  part  at  least, 
the  creation  of  an  Informed  electorate  (or  management).  That  goal  will 
not  be  served  by  a  procedure  that  uncritically  accepts  people's  misin¬ 
formed  Ideas  about  their  own  values. 

The  resolution  of  this  probltjn  would  seem  to  take  one  outside  the  narrow 
confines  of  formal  decls Ion-making  methods.  One  needs  meta-decisions  on 
questions  like:  Which  of  several  possible  Inconsistent  values  Is  to  be 
accepted?  How  much  education  and  Involvement  Is  needed  before  people 
can  be  treated  as  though  they  are  expressing  their  own  values?7  When 
choosing  questions,  should  axiomatic  acceptability  be  abandoned  for  the 
sake  of  Intuitive  appeal  and  ease  of  response?  When  parties  disagree  on 
an  Issue,  Is  It  fair  to  adopt  a  procedure  which  Imposes  one  perspective 
so  strongly  that  people  are  Impelled  to  agree  (perhaps  with  a  value  that 
none  of  th«  likes)? 

A  decision  analysis  that  explicitly  faced  such  Issues  would  be  much 
messier  than  those  one  usually  finds  today.  However.  It  would  be  some¬ 
what  better  protected  from  the  possibility  of  the  whole  enterprise 
collapsing  under  the  cimulatlve  weight  of  the  Issues  of  value  lability 
which  It  otherwise  Ignores  or  flnr-ses.  The  "new”  decision  analysis 
would  probably  Include  an  explicit  acknowledgement  of  the  artful  use  of 
a  variety  of  questions  and  the  gentle  development  of  respondents' 


'Perhaps  the  only  way  to  ensure  meaningful  citizen  participation  In 
public  policy  Issues  Is  to  Impanel  a  representative  group  of  citizens, 
like  a  Jury,  to  follow  an  Issue  through  the  various  stages  of  debate, 
deliberation  and  clarification. 


opinions,  both  of  which  characterize  the  actual  practice  of  the  "old" 
decision  analysis  In  the  hands  of  Its  best  practl tloners. 

All  elldtors,  be  they  decision  analysts  or  students  of  Jud^nent,  deci¬ 
sion  making,  choice  or  attitudes,  must  decide  at  some  point  whether  or 
not  they  have  adequately  captured  their  respondents'  values.  The  usual 
criteria  are  reliability  and  Internal  consistency  (e.g. ,  transitivity). 
However,  where  the  task  Is  poorly  understood  because  of  complexity  or 
unfaml llarlty  (e.g.,  preferences  for  shades  of  gray),  consistency  of 
response  within  a  given  experimental  mode  may  tell  us  little  beyond  the 
power  of  that  mode  to  Impose  a  particular  perspective  or  generate  a 
consistent,  coping  heuristic. 

Insight  Into  people's  values  may  come  rather  from  posing  diverse  ques¬ 
tions  In  the  hopes  of  eliciting  Inconsistent  responses.  If  situation- 
specific  cues  play  a  large  role  In  determining  what  people  express  as 
their  values.  It  Is  the  variance  In  Judgnent  between  situations  which 
reveals  what  those  cues  may  be.  Therefore,  one  would  want  to  start  the 
study  of  values  with  methodological  pluralism  (Royce.  1975)  or  even 
Dadaism  (Feyerabend,  1975)  designed  to  elicit  the  broadest  range  of 
variation  In  expressed  values.  With  a  large  set  of  possible  determinants 
of  value  In  hand,  one  can  then  try  to  establish  their  salience,  potency 
and  prevalence.  This  approach  has  the  admirable  property  of  (potentially) 
turning  past  morasses  Into  silk  purses,  for  any  set  of  Inconsistent 
results  becomes  a  possible  source  of  systematic  variance.  Inconsistency 
In  values  Is  treated  as  a  success  rather  than  a  failure  of  measurement, 
for  It  Indicates  contexts  defined  sharply  enough  to  produce  a  difference. 
Indeed,  this  was  the  approach  adopted  by  Poulton  (1968)  In  producing  his 
six  models  for  the  'new  psychophysics." 
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0.  CONCLUSION 


Expressed  values  seem  to  be  highly  labile.  Subtle  changes  in  elicita¬ 
tion  mode  can  have  marked  effects  on  what  people  express  as  their  pre¬ 
ferences.  Some  of  these  effects  are  reversible,  others  not;  some  deepen 
the  respondent's  perspective,  others  do  not;  some  are  induced  deliber¬ 
ately,  others  are  not;  some  are  specific  to  questions  of  value,  others 
affect  Judgnents  of  all  kinds;  some  are  well  docianented,  others  are 
mere  speculation.  Confronting  these  effects  is  unavoidable  if  we  are 
to  elicit  values  at  all. 

To  the  extent  that  these  effects  are  real  and  powerful ,  they  have 
different  implications  for  different  groups  of  el  id  tors. 

If  one  is  Interested  in  how  people  express  their  values  in  the  real 
world,  one  question  may  be  enough.  That  world  often  asks  only  one  ques¬ 
tion  (e.g. ,  in  a  ballot  measure).  A  careful  analysis  of  how  an  Issue  is 
posed  may  allow  one  to  identify  that  question  and  accurately  predict 
responses. 

If  one  Is  Interested  in  how  people  create,  revise  and  express  their 
opinions,  the  contrast  between  different  procedures  may  be  a  source  of 
insight. 

If  one  is  Interested  In  what  people  really  feel  about  a  value  issue, 
there  may  be  no  substitute  for  an  interactive,  dialectical  elicitation 
procedure,  one  that  acknowledges  the  elicitor's  role  In  helping  the 
respondent  to  create  and  enunciate  values.  That  help  would  include  a 
conceptual  analysis  of  the  problem  and  of  the  personal,  social,  ethical 
value  Issues  to  which  the  respondent  might  wish  to  relate. 


The  most  satisfying  way  to  Interact  with  our  respondents  and  help  them 
make  value  Judgments  In  their  own  best  Interests  Is  to  provide  them  with 
new  analytical  tools.  Such  tools  would  change  respondents  by  deepening 
their  perspective.  In  the  extreme,  they  could  include  relevant  Instruc¬ 
tion  In  philosophy,  economics,  sociology,  anthropology ,  etc.,  as  well  as 

Q 

training  In  decision-making  methodology.  More  modestly,  one  could  con¬ 
vey  an  understanding  of  the  basic  models  for  values  (compensatory ,  dis¬ 
junctive,  etc.),  of  useful  heuristics  (and  their  limitations),  of 
comnonly  accepted  rules  of  rationality  and  their  rationales,  of  common 
pitfalls,  and  of  new  concepts  encountered  In  a  particular  problem. 
Perhaps  the  simplest  and  most  effective  message  of  all  might  be  the 
theme  of  this  paper:  consider  more  than  one  perspective. 


zeboom  (1977)  has  argued  that  the  ellcltors  themselves  should  have 
more  of  such  training. 
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*  ‘Decision  making  requires  two  sequential  actions:  assessing  what  one 
wants  and  then  figuring  how  best  to  obtain  that  goal.  Typically  It  Is 
assumed  that  people  know  what  they  want  and  that,  with  a  little  care, 
those  values  can  be  elicited.  Furthermore,  It  Is  assumed  that  people's 
values  are  sufficiently  well  developed  to  be  elicited  In  whatever  form  Is 
most  convenient  for  the  ellcltor.  Research  reviewed  here  suggests  that 
this  may  not  be  the  case  and  that  a  fundamental  change  may  be  called  for 
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"The  elicitation  procedures  used  may  be  a  major  factor  In  shaping 
the  values  of  people  who  do  not  hold  well -formula ted  values.  An  ellcltor 
can  Induce  random  error,  systematic  error  or  unduly  extreme  Judgments. 
There  may  be  no  substitute  for  an  Interactive,  dialectical  elicitation 
procedure  that  acknowledges  the  ellcltor's  role  In  helping  to  create  and 
enunciate  values.  Respondents  might  be  provided  with  analytical  tools 
to  deepen  their  perspective  and  help  them  make  values  Judgments  In  their 
own  best  Interests.  However,  there  Is  no  guarantee  that  even  the  greatest 
of  care  will  resolve  fundamental  Inconsistencies  In  values,  and  It  may, 
at  times,  be  necessary  to  acknowledge  that  we  do  not  know  what  we  want 
and  structure  our  actions  accordingly. 
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